Duplicate detection methodology for IP network traffic analysis

摘要

Network traffic monitoring systems have to deal with a challenging problem:the traffic capturing process almost invariably produces duplicate packets. Inspite of this, and in contrast with other fields, there is no scientificliterature addressing it. This paper establishes the theoretical backgroundconcerning data duplication in network traffic analysis: generating mechanisms,types of duplicates and their characteristics are described. On this basis, aduplicate detection and removal methodology is proposed. Moreover, ananalytical and experimental study is presented, whose results provide adimensioning rule for this methodology.